tree-sitter

What is tree-sitter?

Tree-sitter is a parser generator tool and an incremental parsing library. It is used to build parsers for programming languages and to parse code into syntax trees. Tree-sitter is designed to be fast and efficient, making it suitable for real-time applications like code editors.

What are tree-sitter's main functionalities?

Parsing Code

This feature allows you to parse source code into a syntax tree. In this example, we parse a simple JavaScript code snippet and print the resulting syntax tree.

const Parser = require('tree-sitter');
const JavaScript = require('tree-sitter-javascript');

const parser = new Parser();
parser.setLanguage(JavaScript);

const sourceCode = 'const x = 1 + 2;';
const tree = parser.parse(sourceCode);

console.log(tree.rootNode.toString());

Querying Syntax Trees

This feature allows you to query syntax trees using a pattern-matching language. In this example, we query the syntax tree for binary expressions with an identifier on the left and a number on the right.

const Parser = require('tree-sitter');
const JavaScript = require('tree-sitter-javascript');
const { query } = require('tree-sitter-query');

const parser = new Parser();
parser.setLanguage(JavaScript);

const sourceCode = 'const x = 1 + 2;';
const tree = parser.parse(sourceCode);

const q = query(JavaScript, '(binary_expression left: (identifier) right: (number))');
const matches = q.matches(tree.rootNode);

console.log(matches);

Incremental Parsing

This feature allows you to incrementally parse code, which is useful for real-time applications like code editors. In this example, we first parse a JavaScript code snippet and then update the syntax tree with a modified version of the code.

const Parser = require('tree-sitter');
const JavaScript = require('tree-sitter-javascript');

const parser = new Parser();
parser.setLanguage(JavaScript);

let sourceCode = 'const x = 1 + 2;';
let tree = parser.parse(sourceCode);

sourceCode = 'const x = 1 + 2 + 3;';
tree = parser.parse(sourceCode, tree);

console.log(tree.rootNode.toString());

Other packages similar to tree-sitter

esprima

Esprima is a high-performance, standard-compliant ECMAScript parser. It parses JavaScript code into an abstract syntax tree (AST). Compared to Tree-sitter, Esprima is specifically focused on JavaScript and does not support incremental parsing.

acorn

Acorn is a small, fast, JavaScript-based JavaScript parser. It generates an abstract syntax tree (AST) and is known for its performance and modularity. Unlike Tree-sitter, Acorn is limited to JavaScript and does not support incremental parsing.

README

node tree-sitter

Incremental parsers for node

Installation

npm install tree-sitter

Usage

Create a grammar using tree-sitter-cli. See the JavaScript grammar and the Go grammar for some examples.

Make a document using your grammar:

const {Document} = require('tree-sitter');

const document = new Document();
document.setLanguage(require('tree-sitter-javascript'));
document.setInputString('let x = 1; x++; console.log(x);');
document.parse();

Access the document's AST:

console.log(document.rootNode.toString());

// (program
//   (lexical_declaration
//     (variable_declarator (identifier) (number)))
//   (expression_statement
//     (update_expression (identifier)))
//   (expression_statement
//     (call_expression
//       (member_expression (identifier) (property_identifier))
//       (arguments (identifier)))))

const callExpression = document.rootNode.children[2].children[0];
console.log(callExpression);

// { type: 'call_expression',
//   startPosition: {row: 0, column: 16},
//   endPosition: {row: 0, column: 30},
//   startIndex: 0,
//   endIndex: 30,
//   children: { length: 2 } }